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Abstract 

Background: Risk adjustment is crucial for comparison of outcome in medical care. Knowledge of the external 
factors that impact measured outcome but that cannot be influenced by the physician is a prerequisite for this 
adjustment. To date, a universal and reproducible method for identification of the relevant external factors has not 
been published. The selection of external factors in current quality assurance programmes is mainly based on 
expert opinion. We propose and demonstrate a methodology for identification of external factors requiring risk 
adjustment of outcome indicators and we apply it to a cataract surgery register. 

Methods: Defined test criteria to determine the relevance for risk adjustment are "clinical relevance" and "statistical 
significance". Clinical relevance of the association is presumed when observed success rates of the indicator in the 
presence and absence of the external factor exceed a pre-specified range of 10%. Statistical significance of the 
association between the external factor and outcome indicators is assessed by univariate stratification and multivariate 
logistic regression adjustment. 

The cataract surgery register was set up as part of a German multi-centre register trial for out-patient cataract surgery in 
three high-volume surgical sites. A total of 14,924 patient follow-ups have been documented since 2005. Eight external 
factors potentially relevant for risk adjustment were related to the outcome indicators "refractive accuracy" and "visual 
rehabilitation" 2-5 weeks after surgery. 

Results: The clinical relevance criterion confirmed 2 ("refractive accuracy") and 5 ("visual rehabilitation") external factors. 
The significance criterion was verified in two ways. Univariate and multivariate analyses revealed almost identical 
external factors: 4 were related to "refractive accuracy" and 7 (6) to "visual rehabilitation". Two ("refractive accuracy") 
and 5 ("visual rehabilitation") factors conformed to both criteria and were therefore relevant for risk adjustment. 

Conclusion: In a practical application, the proposed method to identify relevant external factors for risk adjustment 
for comparison of outcome in healthcare proved to be feasible and comprehensive. The method can also be adapted 
to other quality assurance programmes. However, the cut-off score for clinical relevance needs to be individually 
assessed when applying the proposed method to other indications or indicators. 
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Background 

Quality assurance procedures aim to highlight differ- 
ences in the qualify of medical care between providers. 
However, the outcome of medical care is not only deter- 
mined by the medical contribution, but also by factors 
that cannot be influenced by the physician or medical 
care facility (referred to hereafter as "external factors") 
[1-3]. A selective comparison of the contribution of 
medical care to the outcome measured by means of an 
indicator assumes that the effect based on these external 
factors will be negated by risk adjustment [1]. 

In the present study, a methodological approach based 
on two parallel test criteria is introduced to identify ex- 
ternal factors requiring risk adjustment in quality assur- 
ance programmes. These criteria are "clinical relevance" 
and "statistical significance" of the relationship between 
the external factor and the measured outcome. Each cri- 
terion will be operationalised using statistical association 
analysis techniques. 

The proposed method is demonstrated with data from a 
multi-centre register database on outcome quality in 
cataract surgery. The observed outcome indicators are 
"refractive accuracy" and "visual rehabilitation" 2-5 weeks 
after surgery. The external factors considered cover 
sociodemographic and quantitatively and qualitatively 
documentable patient characteristics. 

Methods 

Test criteria and their statistical parameterisation 

The test criteria for identifying external factors requiring 
risk adjustment were derived according to published in- 
formation on external factors in risk adjustment [4-6]. In 
the methodological approach presented here, a factor 
requires adjustment when it shows clinically relevant 
and statistically significant association with the out- 
come indicators being considered for a quality assurance 
programme. 

For parameterisation of the clinical relevance criterion, 
the success rates (relative frequencies) for the indicators 
are determined in the presence and absence of each ex- 
ternal factor. The absolute difference of these frequen- 
cies is interpreted as a measure of the factor's clinical 
relevance for risk adjustment; if this difference exceeds a 
pre-specified cut-off, the relevance for risk adjustment is 
presumed to be demonstrated. 

The criterion "statistical significance" is deemed to be 
met when demonstrated by at least one of two appropri- 
ate methods: 

a) Univariate stratification: As a measure of the 
association between external factor and indicator, 
the Odds Ratio (OR) of the success rates under 
presence versus absence of the external factor can 
be determined. To correct for potentially differing 



frequencies of occurrence among documenting 
centres, the Odds Ratios should be estimated under 
stratification for centres, for example by means of 
the Mantel/Haenszel method. An OR < 1 then 
indicates that with the presence of the factor fewer 
patients achieve success in terms of the outcome 
indicator. To evaluate the statistical significance of 
the association, a local 99% confidence interval can 
be assigned to the OR estimate; if this local interval 
does not include "1", the corresponding external 
factor is significantly associated with the outcome 
quality at the local 1% significance level, 
b) Multiple adjustment: A multiple regression model 
can consider, in addition, the potential interaction of 
several external factors with each other. A (binary) 
outcome indicator can then be related to the set of 
external factors as potential explanatory variables by 
means of a multiple (logistic) regression model. In 
the regression model, the documenting centres can 
also be considered as explanatory variables; the 
association between external factor and outcome 
indicator can therefore be "adjusted" for centre 
differences instead of being stratified for centres as 
above. The multiple adjustment criterion of local 
statistical significance is met if the local 99% 
confidence interval of an external factor's OR does 
not include "1". 

In general, the results of multiple regression modelling 
tend to be more informative than the results of univari- 
ate association analysis; on the other hand, the results of 
multiple regressions should only be applied if there is 
sufficient model fit (i.e. 80% in terms of an appropriate 
goodness of fit estimate). Therefore, it is generally 
recommended that univariate stratification should be 
applied in the first instance to obtain formal informa- 
tion on the significance of an external factor-outcome 
indicator association; additional multiple adjustment 
information should be considered if there is sufficient 
model fit, and this multiple adjustment criterion can 
then be considered as the more informative significance 
criterion. 

Logistic regression models were constituted using 
backward selection based on local Likelihood Ratio tests 
at the nominal 1% level; the overall goodness of fit of a 
model was then estimated by means of Nagelkerke's R^ 
[2] coefficient. All analyses were performed using SPSS'" 
(Version 20.0 for Windows'"). Furthermore, results of the 
logistic regression modelling were validated using the 
public domain Software R. 

The MONIKA database 

The OcuNet Group, an association of eye surgery prac- 
tices in Germany, has been using the register survey 
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"MONItoring in der ambulanten KAtaraktchirugie" 
["MONItoring in outpatient CAtaract surgery"] — 
hereafter MONIKA — since 2005 as an internal quality 
assurance tool: the data are not publicly available. The 
database was originally set up as a quality monitoring in- 
strument for the affiliated centres using regular reports. 
To meet the user's demands in terms of readability an easy 
report systematic was chosen, endpoints and (possible) ex- 
ternal factors were mostly quoted as binarized. 

The database contains only completely anonymous 
data: each health care provider took responsibility to 
fully delete any patient information, which would have 
enabled identification of treated patients. After this 
process of data modification the anonymous site data were 
forwarded and gathered in an appropriate central data 
base. In summary, neither an approval from local inde- 
pendent ethics committees nor a permission from any 
other body or authority was required to constitute the pro- 
ject data contained in the MONIKA data set according to 
German Federal Data Protection Laws, e.g. as stipulated in 
the Berlin State Data Protection Law [7]. The documenting 
outpatient study centres are so-called "high-volume" cen- 
tres with an above-average number of surgical procedures 
and a broad surgical range. The MONIKA database repre- 
sents the clinical routine in the study centres. All patients 
with scheduled cataract surgery are eligible to be included 
in the database, regardless of diagnostic or therapeutic 
methods used. Anonymised patient characteristics, diag- 
nostics, therapy, and type of implant (intraocular lens) are 
documented in the database. 

Individual patient data are collected preoperatively, in- 
traoperatively, and on three occasions postoperatively. 
Prior to surgery, personal characteristics like gender or 
age, visual acuity, axial length, intraocular pressure, tar- 
get refraction, intraocular lens power, previous ocular 
surgery, presence of at least one surgically relevant ocu- 
lar risk factor and presence of preexisting conditions, 
reducing visual acuity were documented. The intraopera- 
tive documentation covered the type of intraocular lens 
and surgical technique and possible complications during 
surgery. All cataract operations were performed by using 
phacoemulsification. On the first postoperative day as well 
as 2-5 weeks and three months later outcome characteris- 
tics including refraction, visual acuity and postoperative 
complications were recorded. 

The following analyses wUl concentrate on documented 
outcome data assessed 2-5 weeks postoperatively. The first 
100 consecutively documented records per centre were 
excluded to eliminate possible learning curve effects. 

Indicators of outcome quality 

In microsurgery of age-related cataracts, the opaque 
crystalline lens is extracted via the so-called phacoemul- 
sification technique (ultrasound) and replaced by an 



artificial intraocular lens in the capsular bag. The out- 
come quality of this surgery is usually measured using 
two outcome indicators: [8,9] "refractive accuracy" and 
"visual rehabilitation". 

"Refractive accuracy" describes the actual deviation of 
the preoperatively intended (target refraction (TR)) to 
postoperatively achieved refractive power of the eye 
(spherical equivalent (SE)). A "good" surgical outcome 
in terms of this indicator is defined as absolute TR - 
SE < 0.5 dpt (corresponding to the fact that most patients 
with this outcome will not need correcting glasses for 
distance vision). 

"Visual rehabilitation" describes the postoperative vis- 
ual acuity. "Success" in terms of this indicator will be 
ascertained when the postoperative visual acuity cc 
(=cum correctione) is at least 1.0, corresponding to the 
physiological visual acuity of young adults. 

External factors considered 

The external factors considered were categorised ac- 
cording to "sociodemographic" and "quantitatively" and 
"qualitatively" documented individual patient charac- 
teristics. Sociodemographic factors included "age at 
surgery > 80" and "female gender"; quantitatively docu- 
mented external factors included "presence of a poor 
baseline visual acuity" (cc < 0.1), "severe nearsightedness" 
(myopia, axial eye length > 25 mm) and "severe farsighted- 
ness" (hyperopia; axial eye length < 22 mm) [10]. Qualita- 
tively documented external factors included the "presence 
of at least one pre-existing condition that (potentially) re- 
duces the visual acuity" or "at least one known previous 
ocular surgery" or the "presence of at least one surgically 
relevant ocular risk factor". 

Patient data 

Between 2005 and 2011, five outpatient eye surgery cen- 
tres of the OcuNet Group documented a total of 19,278 
raw data sets (the first 100 consecutively documented 
data sets per centre were excluded) in the MONIKA 
database. However, the (voluntary and thus fluctuating) 
participation of the centres varied greatly over time: 
completeness of the MONIKA database varies according 
to postoperative examination, centre and observed indi- 
cators of the outcome quality. Most of the data for both 
the refractive and the visual outcome indicator were 
available for the examination 2-5 weeks after cataract 
surgery; the following analyses are therefore based on 
the data for this examination. Furthermore, to demon- 
strate the above methodological approach, only the data 
from three of the five documenting centres will be ana- 
lysed, as two of the centres contributed relatively few 
valid data sets compared to the others. 

A total of 14,924 patients were analysed (Table 1); 
58.6% of these documented patients showed complete 
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Table 1 Absolute frequency of documented raw data 
sets per study centre over time for the consecutive 
documentation of cataract surgery in three centres 





2005 


2006 


2007 


2008 


2009 


2010 


2011 


total 


Centre 1 


55 


1,445 


906 


1,122 


681 


428 


171 


4,808 


Centre 2 






771 


572 


1,038 


137 




2,518 


Centre 3 




1,309 


1,484 


1,222 


1,191 


1,413 


979 


7,598 


Total 


55 


2,754 


3,161 


2,916 


2,910 


1,978 


1,150 


14,924 



information for the postoperative outcome indicator 
"refractive accuracy" and 56.5% for the postoperative out- 
come indicator "visual rehabihtation" at 2-5 weeks. 

Results 

Table 2 summarizes the reported prevalences per centre 
of the eight external factors with putatively relevant 
and significant association with the outcome indicators 
under consideration (thereby necessitating risk adjust- 
ment of success rates for these indicators). Note that for 
most external factors, the prevalences reported by centre 
2 differ from those reported by centres 1 and 3, respect- 
ively. As a consequence, centre-stratified assessment of 
association estimates between these external factors and 
the outcome quality indicators was found to be required 
and reasonable. 

The success rates without risk adjustment (for pres- 
ence of all external factors) were 68.9% for refractive ac- 
curacy (centre range 62.0% to 74.1%) and 47.0% for 
visual rehabilitation (centre range 32.5% to 51.3%) for 
the combined data of the three centres. Table 3 summa- 
rises the success rate estimates [%] of the two indicators 
2-5 weeks after cataract surgery under stratification for 



the presence of each respective external factor. With the 
exception of the factors "female gender" and "age at 
surgery > 80 years", all "refractive accuracy" indicator- 
related success rates are lower in the presence of the 
respective factor than in its absence. For the indicator 
"visual rehabilitation", all success rates were reduced in 
the presence of any external factor. 

Evaluation of the criterion "clinical relevance" 

For refractive accuracy only two out of eight factors 
("baseline visual acuity < 0.1" and "severe myopia") 
met the 10% difference criterion introduced here to 
characterise clinically relevant association between fac- 
tors and outcome indicator. Whereas the external fac- 
tor "severe nearsightedness" did not meet this criterion 
related to the indicator visual rehabilitation, "severe 
farsightedness" did as did most of the other external 
factors with outcome success rate reductions of up to 
20.7%. 

Evaluation of the criterion "statistical significance" 
by stratification 

The centre-stratified Odds Ratios resulting from strati- 
fied association analysis between the eight external fac- 
tors and the respective outcome quality indicators are 
summarised in Table 4: locally significant associations 
(the 99% confidence interval of the respective OR does 
not include the value "1") with the indicator visual re- 
habilitation were found for all external factors con- 
sidered except "severe nearsightedness". However, for 
refractive accuracy, only the quantitatively documented 
factors and the qualitative factor "presence of at least one 
surgically relevant risk factor" showed locally significant 
associations. 



Table 2 Reported prevalences [%] of sociodemographic, quantitatively and qualitatively documented external factors 
with putative relevance for two outcome indicators of cataract surgery (refractive accuracy and visual rehabilitation), 
stratified and cumulated for three outpatient centres 

Centre 1 2 3 1-3 

[%] 

Sociodemographic external factors 

Age > 80 years 28.1 18.7 22.4 23.6 

Female gender 59.2 56.8 60.3 59.4 

Quantitatively documented external factors 

Baseline visual acuity cc< 0.1 144 9.2 7.0 9.8 

Severe nearsightedness (myopia, axial eye length > 25 mm) 7.5 11.0 8.7 8.7 

Severe farsightedness (hyperopia, axial eye length < 22 mm) 8.6 12.6 9.5 9.7 

Qualitatively documented external factors 

Presence of at least one pre-existing condition (potentially) reducing visual acuity 40.8 27.6 32.7 35.4 

Presence of at least one known previous ocular surgery 3.8 3.7 1.8 2.9 

Presence of at least one surgically relevant ocular risk factor 28.1 11.0 26.2 24.9 
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Table 3 Relative frequencies ("success rates" [%]) of two outcome indicators (visual rehabilitation and refractive 
accuracy) 2-5 weeks after cataract surgery, stratified for presence / absence of each putative external factor, based 
on the data of three outpatient centres 







"Refractive accuracy" 
1 SE-TR 1 <0.5dpt 




"Visual rehabilitation" 
visual acuity cc > 1.0 


Absence Presence 


DiffsrsncG 


Absence Presence 


Difffirsncs 




[%] 


[%] 


r CI LCI 1 LaUC 

points 


[%] 


[%] 


r C 1 L C 1 1 La u c 

points 


Sociodemographic external factors 














Age > 80 years 


67.1 


69.4 


- 2,3 


51.6 


33.1 


18.5 


Female gender 


67.9 


69.6 


- 1,7 


49.2 


454 


3.8 


Quantitatively documented external factors 














Baseline visual acuity cc < 0.1 


70.0 


59.6 


10,4 


48.6 


34.3 


14.3 


Severe nearsightedness (myopia; axial eye length > 25 mm) versus 
normal eye length (22.01 - 24.9 mm) 


70.8 


58.6 


12,2 


48.5 


47.1 


14 


Severe farsightedness (hyperopia, axial eye length < 22 mm) versus 
normal eye length (22.01 - 24.9 mm) 


70.8 


62.5 


8,3 


48.5 


33.6 


14.9 


Qualitatively documented external factors 














Presence of at least one pre-existing condition (potentially) reducing 
visual acuity 


69.5 


67.6 


1,9 


48.5 


33.6 


14.9 


Presence of at least one known previous ocular surgery 


69.0 


65.9 


3,1 


47.6 


26.9 


20.7 


Presence of at least one surgically relevant ocular risk factor 


70.0 


65.6 


4,4 


49.0 


41.1 


7.9 


TR: Target refraction, SE: Spherical equivalent 2-5 weeks after cataract surgery. 
Visual acuity cc: best-corrected visual acuity 2-5 weeks after cataract surgery. 



Evaluation of the criterion "statistical significance" by 
centre-adjusted logistic regression 

Logistic regression analysis (Table 5) showed an ex- 
tremely low model fit (Nagelkerke's adjusted for re- 
fractive accuracy 3% and for visual rehabilitation 12%, 



respectively); as a consequence, the decision for risk 
adjustment for the external factors should not take these 
results into account and should be restricted to the uni- 
variate findings of stratified analysis. However, despite 
this low model fit, locally significant association at the 



Table 4 Results of the univariate association analysis (under stratification for centres by means of the Mantel / Haenszel 
method) between putative external factors and two outcome indicators (refractive accuracy and visual rehabilitation) 
2-5 weeks after cataract surgery, based on the combined data of three outpatient centres: Odds ratios (OR) and local 
99% confidence intervals (CI) 

"Refractive accuracy" "Visual rehabilitation" 

I SE - TR I <0.5 dpt visual acuity cc> 1.0 

OR 99% CI OR 99% CI 

Sociodemographic external factors 

Age > 80 versus age < 80 0.885 0.772-1.040 0.484 0414-0.564 

Female gender versus male gender 1.127 0.986-1.276 0.870 0.767-0.979 

Quantitatively documented external factors 

Baseline visual acuity cc < 0.1 versus > 0.1 0.613 0.492-0.777 0.556 0.443-0.688 

Severe nearsightedness (myopia; axial eye length > 25 mm) versus normal eye length (22.01 - 24.9 mm) 0.613 0.486-0.744 0.993 0.805-1.209 
Severe farsightedness (hyperopia, axial eye length < 22 mm) versus normal eye length (22.01 - 24.9 mm) 0.737 0.587-0.919 0.541 0.434-0.672 
Qualitatively documented external factors 

Presence versus absence of at least one pre-existing condition (potentially) reducing visual acuity 0.915 0.804-1.047 0.508 0.442-0.571 

Presence versus absence of at least one known previous ocular surgery 0.855 0.60-1.244 0.393 0.253-0.561 

Presence versus absence of at least one surgically relevant ocular risk factor 0.773 0.675-0.888 0.732 0.632-0.840 

TR: Target refraction, SE: Spherical equivalent 2-5 weeks after cataract surgery. 
Visual acuity cc: best-corrected visual acuity 2-5 weeks after cataract surgery. 
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Table 5 Results of multiple logistic regression analyses to evaluate the association between putative external factors 
(explanatory variables) and two outcome indicators (refractive accuracy and visual rehabilitation) 2-5 weeks after 
cataract surgery based on the combined data of three outpatient centres: Odds ratios (OR) and local 99% confidence 
intervals (CI) for each explanatory variable; underlying model fit assessed by Nagelkerke's adjusted R^ criterion for the 
respective multiple logistic regression models 





"Refractive accuracy" 
1 SE - TR 1 < 0.5 dpt 


"Visual rehabilitation" 
visual acuity cc> 1.0 


Ni^npllfprlcp rmnHnpcc of fit 




0.03 




0.12 




OR 


99% CI 


OR 


99% CI 


Sociodemographic external factors 










Age > 80 years 


0.872 


0.749-1.030 


0.478 


0.409-0.560 


Female gender 


1.120 


0.981-1.289 


0.915 


0.808-1.045 


Quantitatively documented external factors 










Baseline visual acuity cc < 0.1 


0.705 


0.567-0.891 


0.568 


0.433-0.698 


Severe nearsightedness (myopia, axial eye length > 25 mm) 


0.612 


0.498-0.760 


0.958 


0.771-1.200 


Severe farsightedness (hyperopia, axial eye length < 22 mm) 


0.713 


0.591-0.881 


0.579 


0.468-0.729 


Qualitatively documented external factors 










Presence of at least one pre-existing condition (potentially) reducing visual acuity 


1.016 


0.878-1.178 


0.519 


0.449-0.584 


Presence of at least one known previous ocular surgery 


1.005 


0.697-1.542 


0.535 


0.339-0.771 


Presence of at least one surgically relevant ocular risk factor 


0.766 


0.660-0.896 


0.834 


0.730-0.962 


Centre contrasts 










1 versus 3 


0.763 


0.644-0.892 


1.304 


1.110-1.503 


2 versus 3 


0.579 


0.491-0.697 


0.449 


0.360-0.531 



TR: Target refraction, SE: Spherical equivalent 2-5 weeks after cataract surgery. 
Visual acuity cc: best-corrected visual acuity 2-5 weeks after cataract surgery. 



1% level was demonstrated for almost the same external 
factors as those already identified by stratification. In an 
exploratory sense, these regression results do not, in 
general, contradict the findings of the univariate associ- 
ation analysis, but cannot be called upon to confirm 
them. 

Discussion 

In this paper, a method for identification of external fac- 
tors relevant for risk adjustment using quantitative test 
criteria is proposed and demonstrated by using data 
from a multi-centre register database for outcome qual- 
ity in cataract surgery. Risk adjustment serves to estab- 
lish comparability of different medical facilities and is an 
important tool for quality assurance. While methodo- 
logical risk adjustment procedures have already been 
adopted and are well established [11], little is known 
about how to identify the factors relevant for risk adjust- 
ment. Clinical relevance is discussed in some publica- 
tions as a relevant characteristic in addition to statistical 
significance [5,6]. However, published literature does not 
include suggestions for the operationalisation of these 
criteria for specific indications and databases. In the 
present study, test criteria are qualitatively defined and 
parameterised using statistical methods for association 
analysis and by context-specific cut-off proposals. 



Comprehensive analysis for identification of external 
factors relevant for risk adjustment should also consider 
the test criterion "uneven distribution among care pro- 
viders". We did not apply this criterion to the MONIKA 
database since the study centres are structurally similar 
(outpatient specialist providers) and, therefore, do not 
provide a true representation of the distortion in distri- 
bution of external factors throughout different levels of 
healthcare. 

Clinical relevance 

An external factor has clinical relevance if patients are 
expected to experience disadvantages if it is present. As 
a measure of clinical relevance, the (absolute) deviation 
of the success rates with or without presence of the ex- 
ternal factor was utilised in this analysis. Clinical rele- 
vance can be considered to be greater, the higher the 
deviation in success rates in the presence or absence of 
the external factor. In this investigation, taking into ac- 
count the nature of the actual cataract surgery outcome 
indicator, a threshold limit value of at least 10 percent 
points was proposed to classify clinically relevant as- 
sociation between external factors and outcome quality 
indicators. 

However, these cut-off points defining clinical relevance 
are based on normative settings and inherent reliability is 
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not specified. Therefore, cut-off limits must be determined 
according to the actual indicators and their context in 
quality assurance procedures. For example, if an endpoint 
such as mortality is quantified by means of an indicator, a 
notably lower threshold limit value for the deviation in 
prevalences may be appropriate. An example for cataract 
surgery is the occurrence of postoperative endophthalmitis 
as (negative) indicator of the outcome quality, with a re- 
ported incidence between 0.5% and 0.015% [12]. Due to 
the high clinical relevance (high risk of blindness), a minor 
prevalence difference of predisposing external factors 
would be sufficient to declare clinical relevance for risk 
adjustment. In terms of this, utilisation of the above clin- 
ical relevance criterion based on the "absolute" prevalence 
difference must be critically discussed. 

For positive endpoints like those choosen here deter- 
mination of the cut-off value is a weighting and not 
justifiable by objective arguments: A cut-off point for 
clinical relevance of 5% rather than 10% would have 
resulted in two additional confirmed external factors 
("severe farsightedness" and "presence of at least one 
surgically relevant ocular risk factor"). Lowering thresh- 
old means to consider more external factors and im- 
prove therefore the surrounding for a suitable quality 
comparison. The complexity of documentation, analysis, 
error rate and susceptibility to manipulation increase on 
the other hand. 

A relative clinical relevance criterion would be another 
possible parameterization especially in studies with rare 
outcome variable. A relative risk scale might be consid- 
ered in a study looking at relevant external factors re- 
sponsible for rare events, e.g. endophthalmitis. However, 
in an interdisciplinary clinical setting an absolute excess 
risk scale is more acceptable. When being presented 
with an absolute difference most clinicians find such in- 
formation easier to understand and more straight-forward 
to interpret as compared to a relative difference. 

Statistical significance 

In the present study, statistical significance was deliberately 
parameterised by means of two established methodological 
approaches: first, univariate stratification association and 
second, multiple (logistic) regression analysis. Although 
both approaches allow for centre-corrected estimation 
of the association between each external factor and each 
outcome indicator, the univariate analysis of the association 
of external factors and the measured outcome quality did 
not allow simultaneous comparison of the influence of the 
surgical facility. 

On the other hand, whereas logistic regression model- 
ling enabled several external factors to be simultaneously 
related to one outcome indicator, model goodness of fit 
for this approach was crucial: although the univariate 
and the multiple approach identified the same external 



factors as those significantly associated with the refract- 
ive and visual outcome indicators, the extremely low 
model fit < 12% for both models did not allow these 
modelling results to confirm the univariate approach 
in an analysis method perspective. The model fit might 
be acceptable with a (negative) endpoint occurring only 
rarely - e.g. complications after surgery - but not for 
binary, well confined positive endpoints. The type of 
variable - dichotomized instead of continuous - might 
have had negative impact on the model fit. Based on the 
results of this cataract example, we propose applying the 
univariate stratification approach in the first instance, 
despite its rather exploratory nature - and to consider 
the results from the regression modelling approach as 
additional information providing confirmation if there is 
sufficient model fit. It is the task of future studies and 
discussion to estimate to what extent (as measured by 
Nagelkerke's R^) logistic regression modelling can be ex- 
pected to explain the variation in the results of cataract 
surgery or other indications. 

Both approaches must be discussed further with 
respect to multiple testing. Whereas both approaches 
accounted for centre heterogeneity by either stratifica- 
tion or adjustment for centres, both methods estimated 
eight external factor associations with each outcome 
quality indicator. As a consequence, a total of 2 x 8 as- 
sociations were tested for significance, formally requiring 
multiplicity adjustment. However, regarding the explora- 
tory nature of identifying external factors requiring risk 
adjustment in future analysis/reports, the above results 
should be considered in terms of locally significant asso- 
ciation findings instead of formally correcting their con- 
fidence interval levels for multiplicity. Nevertheless, 
researchers should be aware of this "local" significance 
interpretation underlying the above significance criterion 
— and should reduce the number of external factors to 
be considered for risk adjustment. 

Factors relevant for risk adjustment 

The selection of the external factors considered in this 
study is based on information in published literature on 
external factors demonstrating influence on success rates 
[9,13-21]. However, regardless of this discussion, risk ad- 
justment in established quality assurance procedures for 
cataract surgery was carried out only for one indicator 
(visual rehabilitation) and for one factor (pre-existing 
conditions reducing visual acuity) [18,22-24]. The results 
of the analyses described here suggest that this limited 
risk adjustment does not sufficiently ensure a valid 
basis of comparison for the selective medical contribu- 
tion to outcome quality in the context of quality assurance 
procedures. 

The external factors considered in this study are catego- 
rized as sociodemographic and quantitatively measurable 
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and qualitative patient-related characteristics documented 
on the basis of clinical findings. The objective of this cat- 
egorisation was discovery of potential structural differ- 
ences that can be attributed to the factor properties. As 
expected, the differences in prevalence between centres 
are smallest, the less complex and the more standardised 
the survey of a factor is. Identification of external factors 
based on qualitative clinical findings of varying complex- 
ity and standardisation therefore constitutes an intrinsic 
source of bias for appropriate risk adjustment for quality 
assurance procedures. 

Methodological considerations of the data quality of the 
MONIKA database 

Adequate data quality is a prerequisite for transferability 
of the results from the MONIKA database to other qual- 
ity assurance procedures. The time-shifted participation 
of the study centres could be regarded as a potential 
source of bias (Table 1). During the entire survey period 
there was no significant change in diagnostic or thera- 
peutic methods employed, so that a systematic effect of 
the divergent entry period on the data quality seems un- 
likely. The incompleteness of the data could also be 
considered as a source for bias. However, reduced com- 
pleteness has also been reported in other voluntary 
studies. The rates in the three centres of the MONIKA 
database are comparable with those of other publications 
with an equivalent study environment [25]. Lower response 
rates resulting from reduced commitment do not allow a 
conclusion on systematic bias [26]. 

Internal and external consistency of the documentation 

The internal consistency of the frequencies of quantita- 
tive to qualitative factors points to good data quality for 
the three study centres (Table 2). Generally, in these 
centres, a large proportion of elderly patients is associ- 
ated with a large number of pre-existing conditions 
reducing visual acuity (and vice versa). In addition, 
the relationship of low baseline visual acuity and pre- 
existing conditions reducing visual acuity was described 
in all centre cohorts. Lastly, an association between re- 
fraction anomalies ("high myopia" or "high hyperopia") 
and at least one previous ocular surgery or surgically 
relevant ocular risk factors is present in all centres. 

Another approach for evaluation of the data quality is 
the comparison of the reported prevalence of external 
factors according to the MONIKA database (Table 3) 
with the literature. This analysis also confirms good 
documentation quality in the MONIKA database. The 
average age at surgery in the MONIKA database (73 years) 
is in the middle of the range of reported averages (67 years 
[22] to 76 years [23]). This also applies to the reported 
proportion of women (MONIKA: 59.4%; range of other 
publications 53% [27] to 66% [23]). The total prevalences 



documented in the MONIKA database for severe near- 
sightedness and severe farsightedness lie only slightly 
below those of a publication by a German university 
hospital [10]. 

Due to inconsistent definitions in the published litera- 
ture, the comparison of prevalence of qualitative docu- 
mented factors is more difficult. In the MONIKA 
database, the two factors "pre-existing conditions redu- 
cing final visual acuity" and "history of previous ocular 
surgery" are documented without quantification of the 
impact on attainable final visual acuity. In contrast. 
Murphy et al. use "ocular comorbidity that was expected 
to reduce postoperative acuity to 6/12 or worse" [22] as a 
definition and Jaycock et al. the limit "ocular copathol- 
ogy, identified as a reason for a guarded visual prognosis 
in the operated eye" [18]. As expected with this definition 
heterogeneity, the reported frequencies deviate strongly 
and range from 21.2% [22] to 41% [28], as compared to 
the cumulative reported prevalence for both factors re- 
ported in the MONIKA database of 36.1%. 

In summary the data from the study centres show ad- 
equate consistency. The data quality can be considered 
sufficient for evaluation of the test criteria to identify 
relevant external factors for risk adjustment of outcome 
indicators in cataract surgery. 

Conclusion 

In this study, a method is presented for identifying exter- 
nal factors, which is to be used as part of a quality assur- 
ance procedure for outcome indicator-dependent risk 
adjustment. Relevance for risk adjustment is proposed if 
the factor's association with an outcome quality indica- 
tor is found to be clinically relevant and statistically sig- 
nificant. Clinical relevance was assumed if the indicator 
success rate deviates more than 10 percentage points 
with or without adjustment. The method proved to be 
feasible and comprehensible when applied to a multicentre 
register database on outcome quality in cataract surgery. 
External factors with indicator-dependent relevance for 
risk adjustment could be determined. The method can be 
adapted to other quality assurance programms, it might 
also be useful to identify unknown risk factors. However, 
the cut-off score for clinical relevance needs to be individu- 
ally assessed when applying the proposed method to other 
indications or indicators. 
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